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structure. 

Specifically, we investigate measures of synchroniza- 
tion and their associated lengths scales for hidden 
Markov models (HMMs) — a particular class of processes 
that have an internal (hidden) Markovian dynamic that 
produces an observed data sequence. We compare and 
contrast various notions of being synchronized in a set- 
ting where one is given a model and a sequence of ob- 
servations. Several of the length scales described (e.g., 
Markov order and synchronization time) are familiar in 
fields such as physics, stochastic processes, information 
theory, and machine learning, while others (e.g., cryptic 
order) are relatively new and their applicability has yet 
to be fully appreciated. In this work, we provide efficient 
algorithms for their calculation. 

The development proceeds as follows. Section [TTJ de- 
fines the length scales and related properties of interest 
for HMMs. Section [Tll| then introduces and proves several 
results that serve to relate them, give insight into their 
meaning, and make their computation tractable. Next, 



I. INTRODUCTION 

Stochastic processes are often described by the spa- 
tial and temporal length scales over which correlations 
exist. In physics, the range of correlations is a struc- 
tural property, giving the range over which significant 
energetic coupling exists between the system's degrees of 
freedom. In time series analysis, knowing the length scale 
of correlations is key to successful forecasting. In biose- 
quence analysis, the decay of correlations along DNA 
base pairs determines in some measure the difficulty faced 
by a replicating enzyme as it "decides" to begin copying 
a gene. The common element in these is that the cor- 
relation scale determines how quickly the observer (ana- 
lyzer, forecaster, or enzyme) comes to synchronize to the 
process — that is, comes to know a relevant structure of 
the stochastic process. 

We recently showed that there are a number of distinct, 
though related, length scales associated with synchroniz- 
ing to presentations of stationary stochastic processes [T] . 
Here, we show that these length scales come in two fla- 
vors: topological and statistical. The topological length 
scales depend only on the underlying graph topology of 
the effective states and their transitions, while the statis- 
tical length scales vary with the underlying probabilistic 



Sec. IV provides efficient methods for computing these 
length scales. By way of illustration, Sec. [V] presents a 
survey of Markov and cryptic orders among finite-state 
e-machines — a unique and privileged representation of a 
process — which shows that infinite correlation is a dom- 
inant property in the space of processes and that it is 
generically hard to exactly synchronize to stationary pro- 
cesses. Finally, Sec. |VI| concludes by suggesting applica- 
tions for which these algorithms will prove useful. 
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II. BACKGROUND 

We assume the reader has an introductory knowledge 
of information theory and finite-state machines, such as 
that found in the first few chapters of Ref . [2] and Ref . [3] , 
respectively. Furthermore, we make use of e-machines, a 
particular representation of a process that makes many 
properties directly and easily computable; see Ref. g]. 
A cursory understanding of symbolic dynamics, such as 
that found in the first few chapters of Ref. [5], is useful 
for several of the results. 



A. Processes 

We denote subsequences in a time series as X a: b, 
where a < b, to refer to the random variable sequence 
X a X a+ iX a+ 2 ■ ■ • which has length b — a. The past 

X is then X-oo-.q and the future X is Ao :00 - We gen- 
erally use w to refer to a word — a sequence of symbols 
drawn from an alphabet A. We place two words, u and v, 
adjacent to each other to mean concatenation: w = uv. 
We define a process to be a joint probability distribution 
Pr( X) over "5? = A^A*. 

A presentation of a given process is any state-based 
representation that generates the process. A pro- 
cess's e-machine is its unique, minimal unifilar presen- 
tation [TS]. The recurrent states of a process's e-machine 
are denoted S and the recurrent states of any other pre- 
sentation are denoted 1Z [16] . Distributions over states 
which occur infinitely often are denoted {7Z}; these are 
the recurrent state distributions. 

Following Ref. [1], we seek to give our definitions 
and algorithms flexibility by allowing generalizations 
of the following definitions to all of a process's non- 
e-machine presentations. Fortunately, our definitions 
change only slightly. All asymptotically synchronizing 
presentations |B] allow essentially the same definitions, 
with causal states replaced by the presentation's states. 
Unifilar presentations with positive gauge information 
PQ, however, require that the definitions replace states 
with asymptotically recurrent state-distributions. (The 
latter are the recurrent states of the mixed-state presen- 
tation; see Ref. [7].) For example, the synchronization 
order can never be finite in a presentation with positive 
gauge information, but if we consider the state distri- 
butions that are visited infinitely often as the "state" 
to which we synchronize, the definition retains its spirit. 
Nonunifilar presentations have no such simple generaliza- 
tion. We summarize the state of affairs in Table U 

For the remainder of this paper we use causal states 
S in all definitions, propositions, theorems, proofs, and 
algorithms for convenience. However, except where ex- 



Presentation Class 


Generalization Rule 


Asymptotically Synchronizing 
Nonsynchronizing Unifilar 
Nonunifilar 


S -> K 
S^{K} 
Unknown or N/A 



TABLE I: Brief guide to generalizing the synchronization def- 
initions to wider classes of presentation. 



plicitly stated as applying only to e-machines, these can 
be exchanged for the states of an alternative model ac- 
cording to Table [I] 

B. Minimal Synchronizing Words 

For the synchronization problem, we consider an ob- 
server who begins with a correct model (a presentation) 
of a process. The observer, however, has no knowledge 
of the process's internal state. The challenge is to an- 
alyze how an observer's knowledge of the internal state 
changes as more and more measurements are observed. 

At first glance, one might say that the observer's 
knowledge should never decrease with additional mea- 
surements, corresponding to a never-increasing state un- 
certainty, but this is generically not true. In fact, it is 
possible for the observer's knowledge (measured in bits) 
to oscillate with each new measurement. The crux of the 
issue is that additional measurements are being used to 
inquire about the current state rather than the state at 
some fixed moment in time. 

It is helpful to identify the set of words that take the 
observer from the condition of total ignorance to exactly 
knowing a process's state. First, we introduce what we 
mean by synchronization in terms of lack of state uncer- 
tainty. Second, we define the set of minimal synchroniz- 
ing words. 

Definition 1. A word w of length L is synchronizing if 
the Shannon entropy over the internal state, conditioned 
on w, is zero: 

Sync(w) & H[S e \X :t = w] = , (1) 

where Sync(w) is Boolean function. 

Definition 2. A presentation's set of minimal synchro- 
nizing words is the set of synchronizing words that have 
no synchronizing prefix: 

£ sy nc = {w | Sync{w) and, 

-^Sync(u) for all u : w = uv} . (2) 

Remark. £ syn c is a prefix-free, regular language. If 
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each word is associated with its probability of being ob- 
served, we obtain a prefix-free code encoding each path to 
synchrony — a word in £ sync — with the associated proba- 
bility of synchronizing via that path. These codes are gen- 
erally nonoptimal in the familiar information-theoretic 
sense. 

C. Markov Order 

Of the length scales to be surveyed, a process's Markov 
order is probably the most widely studied. It measures 
the memory in a time series and roughly corresponds to 
the length of the longest correlations that exist. 

Definition 3. The Markov order R is the number of 
observations required to predict the future as accurately 
as if using the infinite past. 

R = min{^ | Pr(^0C) = Pr(^|-XL& )} . (3) 

If this condition does not hold for any finite I, the Markov 
order is oo. 

A further interpretation of the Markov order is as the 
length at which a process's block-entropy curve H[Xq : i] 
reaches its asymptote [T]: 

R = min{^ | H[X 0U ] = E + th^} . (4) 

In short, R is the length of data one must see before be- 
ing able to predict upcoming symbols with the minimum 
error rate. 

D. Cryptic Order 

The cryptic order [8] has several differing interpre- 
tations, but is best thought of as the number of observa- 
tions required to account for the asymptotic amount of 
state information not transmitted to the future. 

Definition 4. The cryptic order k x J9$ is the number 
of presentation states that cannot be retrodicted given the 
infinite future: 

k x = mm{£\H[S e \x'} = 0} . (5) 

Once again, if this condition does not hold for any finite 
I, then k x = oo. 

This can also be seen as the length at which the block- 
state entropy H[Xq.j,Si] reaches its asymptotic behavior 
PQ. We can therefore understand this as a model-centric 
parallel of the Markov order statement in Eq. Q: 

k x =mm{t\H[X .. i ,S t ]=K + £h ll } . (6) 



E. Synchronization Order 

According to Sec. |II B| one is synchronized to a pro- 
cess's presentation after seeing word w if there is com- 
plete certainty in the state. We now expand this view 
slightly to ask about synchronization over all words of a 
particular length. Equivalently, we examine synchroniza- 
tion to an ensemble of process realizations. 

Definition 5. The synchronization order ks JJ^ is the 
minimum length for which every word is a synchronizing 
word: 

k s = mm{£\H[S i \X O:e }=0} . (7) 

As for the Markov and cryptic orders, fcs *s considered 
oo when the condition does not hold for any finite I. 

F. The Synchronization Distribution 

Taking a slightly more general view than the syn- 
chronization order, we consider statistical properties of 
synchronization, rather than just the absolute length at 
which an ensemble will all be synchronized. In this vein, 
we dehne a distribution that gives the probability for a 
word to first synchronize at length t. 

Definition 6. The synchronization distribution S gives 
the probability of synchronizing to a presentation at 
length I: 

s(e) = Y,PT(w)6(\w\-e) . (8) 

where 5 is the Kronecker delta function. 

Remark. S is normalized: YlfLp S(£) = 1. 

We now draw out two particular quantities from this 
distribution — quantities that have observable meaning 
for a presentation. 

Definition 7. The synchronization time r £0 [77J| / is the 
average number of observations needed to synchronize to 
a presentation: 

T = M Pr M (9) 

= Et[S(t)], (10) 

where the second equality shows that t is also equal to the 
expectation value of the synchronization distribution. 

The synchronization time is useful for understanding 
how long it takes on average to synchronize to a model. 
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This is as opposed to the Markov order which is the mini- 
mal longest-synchronization-time across all presentations 
of a process. 

Definition 8. The synchronization entropy H sync is the 
uncertainty in the synchronization distribution: 

H sync = H[S(£)] . (If) 



Remark. Note that this is quite distinct from the syn- 
chronization information S of Ref. jltf : 

oo 

S = Y / H[S e \X . J ] . 

t=\ 

The synchronization entropy, in contrast, measures 
the flatness of the synchronization distribution. And, 
since the synchronization distribution decays exponen- 
tially with length, the fatter the tail, the higher the un- 
certainty in synchronization. 

III. RESULTS 

We now provide several results related to these length 
scales that shed light on their nature introducing con- 
nections and simplifications that make their computation 
tractable. 

Proposition 1. The synchronization order is: 

k$ = max{R,k x } . (12) 



Proof. First, note that: 

H[Si\X 0: e] = H[X .4,Se\ — H[X . e ] . (f3) 

Since the block-state entropy upper bounds the block 
entropy, the conditional entropy above can only reach 
its asymptotic value once both terms have individually 
reached their asymptotic behavior. The latter are con- 
trolled by k x and R, respectively. □ 

This result reduces the apparent diversity of length 
scales, eventually allowing one to calculate the Markov 
order via the synchronization order, which itself is di- 
rectly computable. 

Proposition 2. For e-machines: 

R = k s . (14) 



Proof. Applying the causal equivalence relation ~ £ to 
Def. [J] we find: 

Pr^lA 7 ) =Pr(AV-«)) => £~ e X_ H:0 . (15) 

This further implies that the causal states <S are com- 
pletely determined by X— R: q: 

H[S \X- R:0 ] = . (16) 

This statement is equivalent to the Markov criterion. □ 

Remark. This provides an alternate proof that the cryp- 
tic order k x is bounded above by the Markov order R in 
an e-machine via a simple shift in indices: 

H[So\X_ R .. ]=Q (17) 
H[S R \X 0:R ] = (18) 
=> H[S R \2] = . (19) 

This proposition gives indirect access to the Markov or- 
der via a particular presentation — the e-machine. Since 
the Markov order is not defined as a property of a pre- 
sentation it would generally be unobtainable, but due to 
unique properties of the e-machine, it can be accessed 
through the synchronization order. 

There is a subclass of e-machines to which one syn- 
chronizes in finite time; these are the exact e-machines of 
Ref. P2]. 

Proposition 3. Given an exact e-machine with finite 
Markov order R, the subshift of finite type that underlies 
it has a "step" J5\/ equal to R. 

Corollary 1. Given an exactly synchronizing e-machine, 
the underlying sofic system is a subshift of finite type if 
and only if R is finite. 

Remark. A process with infinite Markov order can have 
a presentation whose underlying sofic system is a subshift 
of finite type. 

These results draw out a connection with length scales 
of sofic systems from symbolic dynamics. Subshifts of 
finite type have a probability-agnostic length scale analog 
of the Markov order known as the 'step" [5J- In the case 
of e-machine presentations, they are in fact equal. 

We will now show that two of the lengths defined — 
the cryptic and synchronization orders — are topological. 
That is, they are properties of the presentation's graph 
topology and are independent of transition probabilities 
so long as changes to the probabilities do not remove 
transitions and do not cause states to merge. Addition- 
ally, due to Prop. [2j the Markov order can be thought 
of as topological. All three are topological since they de- 
pend only on the length at which a conditional entropy 
vanishes, not on how it vanishes. 



5 



Theorem 1. Synchronization order ks is a topological 
property of a presentation. 

Proof. Beginning from Def. [3| there is length £ = ks at 
which: 

H[St\X 0U ] = PrHH[S e \X 0:e =w] = . 

w£A e 

Thus, H[Si\Xo-£ = w] = for all w € A e , the set of 
length-£ words with positive probability. Since every word 
of length £ is synchronizing, £ is certainly greater than 
the synchronization order. As synchronizing words are 
synchronizing regardless of their probability of occurring, 
the synchronization order ks is topological. □ 

Corollary 2. Markov order R is a topological property 
of an e-machine. 

Proof. Since ks is a topological property by Thm. [7] and 
since an e-machine 's R — ks by Prop. [1| the Markov 
order is topological. □ 

Theorem 2. Cryptic order k x is a topological property 
of a presentation. 

Proof. Beginning from Def. Q there is a length £ = k x 
at which: 

= H[S e \X*} 

= y^ Pr ( w ^ a w)H[Sj\X .\ w \ =w,S\ w \ =<r w ] . 

Here, step (1) simply expands the conditional entropy. 
Step (2) is true provided that the sum is over minimal 
synchronizing words and o~ w is the state to which one syn- 
chronizes via w. This final sum is zero only if the sum 
vanishes term-by-term. Thus, given a word that synchro- 
nizes and the state synchronized to, each term provides a 
cryptic-order candidate — the number of states that could 
not be retrodicted from that state and word. Finally, the 
longest such cryptic order candidate is the cryptic order 
for the presentation. □ 

Restated, the cryptic order k x is topological as it de- 
pends only on the minimal synchronizing words, which 
are topological by definition. 

IV. ALGORITHMS 

We are now ready to turn to the primary task — 
computing the various synchronization length scales 
given a presentation. While all of these algorithms have 



compute times that are exponential in the number of ma- 
chine states, we find them to be very efficient in prac- 
tice. This is particularly the case compared to naive 
algorithms to compute these properties. For example, 
computing synchronization, Markov, or cryptic orders by 
testing successively longer blocks of symbols is exponen- 
tial in the length of the longest block tested. Worse, in 
the case of non-Markovian/oo-cryptic processes the naive 
algorithm will not halt. In addition, the naive implemen- 
tation of Thm. [2] given in the proof to compute the cryp- 
tic order has a compute time of 0(2 2 ), whereas the one 
presented below is a simple exponential of N . 

Unsurprisingly, given the results provided in Sec. |III| 
we begin with the minimal synchronizing words as they 
are the underpinnings of the synchronization and cryp- 
tic orders. The algorithms make use of standard proce- 
dures. Most textbooks on algorithms provide the neces- 
sary background; see, for example, Ref. [13]. 

A. Minimal Synchronizing Words 

We construct a deterministic finite automaton (DFA) 
that recognizes £ S ync of a given presentation M = 
(Q, E), where Q are the states and E are the edges. This 
is done as follows: 

Algorithm 1. 

1. Begin with the recurrent presentation M.. 

2. Construct M. 's power automaton 2 M , producing a 
DFA T = 2 M . 

3. Set the node in T that corresponds to all Ai 's states 
as T 's start state. 

4- Remove all edges between singleton states of T ■ 
(These are the edges from M.) 

5. Set all singleton states of T as accepting states. 

Now, we enumerate £ S ync via an ordered breadth-first 
traversal of T, outputting each accepted word. 

B. Synchronization Order 

Thanks to Eq. ([I]) we see that ks is the shortest length 
t that encompasses all of £ sync . This is, trivially, the 
longest word in £ S ync- With this, computing the syn- 
chronization order reduces to: 

Algorithm 2. 

1 . If £ S ync is infinite, return 00 . 
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2. Enumerate each word in £ S ync and return the length 
of the longest word. 

The test in the first step can be done simply by running 
a loop-detection algorithm on DFA T. If there is a loop, 
then £ sync is infinite. 

C. Markov Order 

Due to Thm. [5J a process's Markov order can be com- 
puted by finding the synchronization order of the pro- 
cess's e-machine. If one does not have the e-machine for 
a process, but rather some other unifilar presentation, 
it is still possible in some cases to obtain the Markov 
order through the synchronization order. That is, the 
algorithms for fcs arid k x provide probes into the presen- 
tation's length scales. It can be the case that R is accessi- 
ble to those probes, if k x < fcs, but it is only guaranteed 
to be accessible in the case of e-machines. Note, there 
exist techniques for constructing the e-machine from any 
presentation [7J. 

D. Cryptic Order 

The cryptic-order candidates of Thm. [2] need not be 
computed individually; they can be computed in par- 
allel by modifying the topological transient structure 
provided by the power automaton. One then retrod- 
icts on the entire graph structure. In the following al- 
gorithm T refers to the power automata of the ma- 
chine Ai. The states p, q, and r of T are elements 
of the power set of the states of M.. By the predeces- 
sors of a state q along edge p A q we refer to the set 
p' = {m|(m A n) G Ai and m G p and n £ q}. These 
are the states mGp which actually transition to a state 
n G q on symbol x. By subset construction below we 
refer to the standard NFA-to-DFA conversion algorithm 

El- 

Algorithm 3. 

1. Construct the power automaton T = 2 . 

2. Add each edge p A q in T to a queue. 

3. For each edge p A q in the queue: 

(a) If edge has already been processed: 

i. Restart loop with next edge in the queue. 

(b) Find the predecessors p' of q along p4i;. 

(c) If\p'\ > 1 andp'^p: 

i. Remove edge p — > q. 



ii. Perform the subset construction on p' 
(implicitly adds the edge p' A q). 

in. For each r A p: 

A. Add edge r —} p' . 
iv. Mark p A q and p' A q as processed. 

(d) Add all edges created in steps S.c.ii and 
S.c.iii.A to the queue. 

The result is an automaton. The longest path in it 
from a transient state to a recurrent is the cryptic or- 
der. Skipping previously processed edges is important 
since for some topologies the algorithm can enter a cycle 
where it will remove and then later add the same edge, 
ad infinitum. 

This algorithm for computing the cryptic order only 
holds for unifilar presentations. 

E. The Synchronization Distribution 

There arc two methods to compute the synchronization 
distribution. The first requires an e-machine with finite 
recurrent and transient components. The second, only a 
finite recurrent component. We present the former case 
first. 

Algorithm 4. 

1. Perform an ordered breadth-first traversal of the fi- 
nite e-machine. 

2. While traversing, keep track of the word induced by 
the path and the product of the probabilities along 
that path. 

3. When a recurrent node is reached, stop that partic- 
ular thread of the traversal. 

4- Sum the probabilities of all words with the same 
length. 

This algorithm produces each minimal synchronizing 
word and its probability in lexicographic order. Then 
words of each length can be grouped and their probabil- 
ities summed to get the synchronization distribution. 

The second algorithm is used when an e-machine with 
finite transient structure is not available: 

Algorithm 5. 

1. Produce all the minimal synchronizing words from 
the DFA given in Algorithm^ 

2. For each minimal synchronizing word, compute its 
probability using the recurrent e-machine J^j. 
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FIG. 1: The distribution of Markov order R and cryptic or- 
der k x for all 1,132,613 six-state, binary-alphabet, exactly- 
synchronizing e-machines. The marker size is proportional to 
the number of e-machines within this class at the same (R, 
k x ) value. 

3. Sum the probabilities of all words with the same 
length. 

Once the distribution is computed using one of the 
above algorithms, it is trivial to compute the Shannon 
entropy and mean of the distribution to get the synchro- 
nization entropy and synchronization time, respectively. 

V. GENERICITY OF SYNCHRONIZATION 

We illustrate applying the above results and algorithms 
by empirically answering several simple questions. How 
typical are infinite Markov order and infinite cryptic or- 
der presentations in the space of finite-state processes? 

Restricting ourselves to e-machines, we enumerate all 
binary processes with a given number of states to which 
one can exactly synchronize [14] . Using these e-machincs 
one can compute, as we just showed, their Markov and 
cryptic orders. The result for all of the 1,132,613 six- 
state e-machines is shown in Fig. [T] 

The number of e-machines that share a (R, k x ) pair is 
illustrated by the size of the circle at that (R, k x ). It is 
easily seen that the vast majority of processes — in fact, 
98% — are non-Markovian at this state-size (6). Further- 
more, most (85% to be exact) of those non-Markovian 
processes are also oo-cryptic. One concludes that, in the 
space of finite-state processes, infinite-range correlation 
and infinitely diffuse internal state information are the 
overwhelming rule. As a consequence, it is generically 
difficult to synchronize. 

Also of interest are the "forbidden" (R, k x ) pairs 
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AH[X 0:e ] - h„ 
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k x R 
block length I 

FIG. 2: Entropy convergence curves versus block length i for 
a family of processes with the same topology, but different 
transition probabilities. The each linear asymptotic behavior 
subtracted out of each curve (see inset). The Markov and 
cryptic orders (the lengths at which the blue (dark) and green 
(lighter) lines, respectively, become flat) are independent of 
the selected probabilities. 

within the space of 6-state e-machines. For example, 
e-machines with k x = 4,5,8,10,11 do not occur with 
R = 13. In addition, in the case of infinite Markov order, 
but finite cryptic order, the latter appears to be bounded 
above by k x = 11 despite the fact that larger finite cryp- 
tic orders exist for finite Markov order processes. 

Since Markov and cryptic orders are topological, they 
are independent of the probabilities assigned to edges in 
a presentation. This can be seen quite dramatically by 
assigning random probabilities to the edges in a presenta- 
tion and plotting the resulting block and block-state en- 
tropies (actually, their scaled derivatives). The result is 
shown in Fig. [2j Note that, over a range of randomly sam- 
pled transition probabilities and therefore over a range of 
stochastic processes, convergence occurs at the exactly 
same lengths. 

Finally, we survey the distribution of synchronization 
times t and synchronization entropies H sync for all 1, 388 
four-state, binary-alphabet, exact e-machines with uni- 
form outgoing transition probabilities. See Fig. [3] It is 
interesting to note that there is structure in the distri- 
bution in the form of veils. However, the veils are not 
the entirety of the distribution, there are many machines 
that fall elsewhere. 



VI. CONCLUSION 

We demonstrated how to compute a number of length 
scales for hidden Markov models, most of which can be 
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ing the process's e-machine. While other properties can 
be computed using other presentations, they are typically 
minimized when the presentation used is the e-machine. 
In addition, the e-machine provides an exact method for 
computing the Markov order whereas, in general, the 
Markov order can only be estimated. It is important 
to stress however that this is not a silver bullet. Given 
a time series, the e-machine must still be estimated to 
compute the Markov order. Often, however, one can ana- 
lytically calculate a process's e-machine. In this case, the 
preceding gives exact results, as illustrated in our survey 
of Markov and cryptic orders for processes described by 
six-state e-machines. 



synchronization time r 



FIG. 3: Distribution of synchronization time r and syn- 
chronization entropy H sync for all 1,388 four-state, binary- 
alphabet, exactly-synchronizing e-machines with uniform out- 
going transition probabilities. Individual histograms for each 
property are shown above and to the right. 



motivated in terms of a process's, or its associated pre- 
sentations', synchronization properties. Interestingly, we 
found that one of the most fundamental and important 
properties — the Markov order R — is computable only us- 
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